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ABSTRACT 

The purpose of this study is to evaluate the accuracy 
of a procedure used to compute an estimate of the lower 
iOO (1-y ) % confidence limit for reliability of a system of 
independent components connected in logical series. The 
procedure takes a Bayesian approach and uses test data on 
the individual components where the sample sizes may be 
unequal and no knowledge of the component failure distri- 
bution is needed. A computer simulation is used to gener- 
ate test failure data and to compute estimates for the 
lower 100 (1-y )% confidence limit on system reliability. 
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I. INTRODUCTION 



The problem of determining economically a lower 
lOO(l-y)^ confidence limit on the system reliability of 
complex and expensive systems has plagued reliability 
managers for a long time. In 1968 Joseph Bram of the Cen- 
ter for Naval Analyses developed an approximate procedure 
to attack this problem for systems of independent components 
connected in logical series."'' It is the purpose of this 
study to test the accuracy of Bram's procedure and to de- 
termine the limits of its usefulness, especially in appli- 
cations involving systems of ten or more components. The 
accuracy measurements are determined through a computer 
simulation. 

In an effort to demonstrate reliability goals on expen- 
sive systems, it is usually not economically feasible to 
test the entire system many times. Bram takes a Bayesian 
approach using test data, successes and failures, on indi- 
vidual components or subsystems to obtain an estimate for 
the lower confidence limit on the overall system reliabil- 
ity. The method assumes that the components are indepen- 
dent, and test sample sizes for the various components may 
be unequal. No knowledge or assumptions about the compon- 
ents’ failure rates is needed for the computation, only 
the test failure data. 

"'"Center for Naval Analyses, O.E.G. Research Contribution 
No. 79 , Confidence Limits for System Reliability , by Joseph 

Bram, 2 February 1968. 
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II. RELIABILITY MODEL AND COMPUTATION PROCEDURES 



A. THE RELIABILITY MODEL AND COMPUTATION PROCEDURE: 

METHOD A 

1. Summary of the Reliability Model 

Consider k independent components, as in Figure 1, 
connect in logical series with respective individual relia- 
bilities p^ , P2 ,...,Pj c . By the product rule for series 
systems, the actual overall system reliability, R , is 
given by 



—CD — 0 — 




Component reliability: 
Number of tests: 

Number of successes: 




P 

n 

s 



k 

k 

k 



FIGURE 1. 

THE SERIES SYSTEM MODEL 



In Bram's Bayesian approach to finding an estimate for 

the lower confidence bound on system reliability, the p^'s, 

and hence R , are considered to be random variables. If the 
s 

di st ribut ion from which the p^ ' s were sampled were known, 
a lower confidence bound could be calculated without 
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testing. Since this distribution is not known however, the 
Bayesian's assume a prior distribution which when combined 
with the test data leads to a posterior distribution, hope- 
fully one that is recognizable and from which confidence 
limits can easily be computed. 

By considering the random variable 

k 

u = - In R = - E In p. 

i=l 1 



the procedure approximates the density of u by 



4» (u) 



B a+1 

a! 



u a e 



-Bu 



The parameters a and B will be estimated from test data. 

This density results from having chosen a prior density for 
p^ which is questionable on the grounds that it is not 
normalizable (see prameterization of the prior density, 

Ref. 1) and leads to a posterior which is also not normaliz- 
able. The choice of priors causes difficulty in estimating 
the parameters of 4>(u) when no failures occur on any of the 
components, and its effects will be investigated later in 
this study. However, since this is an approximate method, 
the development can proceed from the assumption that u does 
in fact possess the above density and that estimates for a 
and B can be determined. 

If a new random variable v is defined where 



v = 2 Bu , 
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then v is distributed "chi square" with 2(a+l) degrees of 

freedom. Then the lower 100(l-y)/£ confidence limit, 

R T r \ > can be found as follows: 

S ,L(y ) 3 



P[2gu < 



2 

X 2(a+1) ,1-y 



] = 



1 



Y 



P[u < = 1 



Y 



P[e -U > 



exp (- 



2 

x 2(a+l) ,1 -y 
26 



)] 



1 - Y 



P[R s > 



exp (- 



2 

X 2(q + 1) ,1 -y 
26 



)] 



1 - Y 



From the probability statement above. 



R s ,L( y ) 



exp(- 



2 

X 2 ( q+1 ) ,1-y 

26 



) 



is the desired approximate lower confidence limit. The 

A 

problem is to find an estimate, R ./ m for R , t \ which 
will yield values sufficiently accurate for the reliability 
manager’s purposes. Bram's method specifies estimates 
a and 6, which are computed from the test data, and substi- 
tuted into the above expression, to obtain 



R T / \ 
s ,L( y) 



exp<- . 4cat . lJ .xl -T ) . 

26 



In the following section Bram's computation procedure for 
a and 6 is described. 
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2 . Description of the Computation Procedure 



In the following, Bram's procedure for computing 
R T / \ , will be referred to as method A, and the author's 

s,L(y)* 

modifications to the original procedure will be referred to 
as methods B and C. 

For descriptive purposes, after testing has been 
done, consider the k system components to be arranged as 
follows: 1, 2 ,..., k^, k^+1,..., k, where components 

1, k^ are those which experience no failures, and 

k^+1, k are those which experience one or more 

failures. Let n^ be the number of tests on component i and 
s^ be the number of successes in the n^ tests. To get 
R s method A proceed as follows: 



M 



A 



k n-^-1 

E E 

i=k 1 +l J = s jL 



1 

j 



V 



A 



k n-j_-l 
1=^+1 o=s. 



1 

7? 3 

J 



a 




1 



M 

- A 

and 6 = — 

V A 



and 



R s,L( y ) 



exp(- i ^ L+l L lz X ) 
2 B 



and are estimates of the mean and variance of u under 
method A. In cases where (a+1) is non-integer, interpolation 
is to be used. 
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3 . Shortcomings of Method A 



It should be noted that the double sums do not 
account for components which experience no failures in 

A 

testing, and if none of the system's components fails, a 
and g are not defined: under these circumstances the pro- 
cedure can not be used as it stands. Herein lies the major 
obstacle to application of the procedure to highly reliable 
system testing programs where it is quite possible that no 
failures will occur. In testing the accuracy of method A, 
whenever no failures occur, R g ) is set equal to unity 
by the author's choice, since Bram makes no allowance for 
this possibility. 

A 

Secondly, as a is defined, it is possible that 
2(a+l), the estimate of the degrees of freedom of the chi 
square random variable can be less than one. Since a chi 
square variate must have at least one degree of freedom, 
some provision must be made to account for this inconsist- 
ency. For computation purposes in testing the original 

A 

procedure of method A and the modifications, whenever 2(a+l) 
is less than one, the degrees of freedom are set equal to 
one by the author's choice. For all fractional values of 
2(a+l) greater than one, linear interpolation between the 
tabulated integer degrees of freedom are used, as specified 
in the original procedure. 

B. FIRST MODIFICATION TO THE COMPUTATION PROCEDURE: 

METHOD B 

The only way that the true lower confidence limit R g ^( Y ) 
can be unity is to have no failures on an infinite number of 
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tests; this would be impossible to demonstrate. In 

searching for a more realistic value for R T / \ when no 

s,L(y) 

failures occur, one might be led to believe that R T/ , 

5 s,L(y) 

should be close to but not equal to one. Due to its 

discrete character, the procedure of method A will compute 

one value of R T , >. which is closest to unity; this will 

happen when one failure occurs on the component with the 

greatest number of tests. An approximation of a more 

accurate estimate of R T , % when no failures occur, can be 

s,L(y) j 

obtained by using one half the original estimates and , 

where one failure occurs on the component with the most 

tests. This has the effect of considering a partial failure 

on the component with the most tests and tends to smooth the 

discrete distribution of values of R , , % near unity. 

s ,L(.y; 

In summary, computations under method B proceed as 
follows : 



M = M 
U B A 



V B ‘ V A 



if < k, and 



m b = 



"\ 



2 (N - 1) 
max 






V B ' 



2 (N - 1)' 
max 



J 



if k 1 = k, 



N is the number of tests on the component with the 
max ^ 



greatest number of tests 
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1 



M 



a = 



B 



V. 



B 



M. 



e = 



B 



'B 



and 



R . = exp[- ^2±lL2zll] 

s,L(y) 2b 



If 2 ( a + 1 ) < 1.0, then set 2(a+l) = 1.0. 



C. SECOND MODIFICATION TO THE COMPUTATION PROCEDURE: 

METHOD C 

In an effort to refine method B so that it might be 
useful under conditions of very high component reliability 
and small amounts of testing, a second modification is 

/V /s. 

presented. Method C of computing M and V is an attempt to 

further smooth the distribution of R T t \ and has the 

s ,L( y ) 

effect of adding a partial failure to each component which 
experiences no failure during testing. For method C compu- 
tation proceeds as follows: 



k l 

Z p + M 

i=l 2 (n i - 1) 



ki 1 

E ± + v 

i=l 2 k^ (n i - l) d 
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a 



1 




> 



6 




and 



R s ,L(y) 



exp [- 



2 . 

X 2(a+1) ,1-y -j 



2 g 



Again, if 2(a+l) < 1.0, set 2(a+l) = 1.0. 



From these computations it is seen that if every 
component experiences at least one failure, i.e. k^ = 0, 
then method C reduces to method B. On the other hand, if a 
system consists of say forty components, and there is 
limited testing relative to the component unreliability of 
say ten of the components, one might feel that some adjust- 
ment should be made to the overall estimate R r , to 

s,L(y) 

account for the lack of testing on those ten components; 
that is the motivation for this second modification. 
Furthermore, any method of computing R g g( Y ) yields 

accurate results with less testing than another method will 
be more desirable economically. 
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III. SIMULATION PROCEDURE 



The purpose of the computer simulation in this study is 
to demonstrate the accuracy of the procedures described in 
section II. The simulation considers the system of k com- 
ponents of Figure 1, where the component reliabilities and 

hence R are assumed to be known. For this system tests are 
s 

simulated by the Monte Carlo technique, generating failure 

data which is used to compute R g each of the three 

methods A, B, and C. By replicating the procedure 1000 

times, 1000 values of R Tf x are generated which are then 

s,L(.y; 

arranged in ascending order. If the method of computing 

R T / x is accurate, then 100(l-y)$ of the time the true 
s,L(y; 

system reliability, R , will be greater than or equal to 

s 

~ f h 

R T / \ • Therefore the lOO(l-y) percentile point of the 
s } Jj \Y ) 

1000 ordered values of R x / \ j A., > should be equal to or 

s ,L ( y ) 1-y * 

very close to the value of R if the method is accurate. 

s 

t h 

For example, if y = .10 then the 9(J percentile point or 

1. A 

900 ordered value of R T/ \ should be close to R„ . It is 
this difference, | R g - A^ | , that is the primary measure of 
accuracy of the method of estimation. Another measure of 
the accuracy is the true level of confidence. The true 
level of confidence is found by determining the percentage 
of the values of R T / \ that are less than or equal to R . 
If the method is accurate the true level of confidence will 
be very close to lOO(l-y)^. 
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One of the main concerns of this study is to determine 
what minimum amount of testing is necessary to insure that 
the method will provide accurate results. This minimum 
amount of testing will be different for each system and will 
be dependent on the number of components and their reliabil- 
ities. For this reason the quantity 

k 

TT = £ n. (1-p. ) 

i=l 1 1 

is used as a measure of the amount of testing relative to 

2 

the system reliability. Low values of TT occur when combi- 
nations of high reliability and small numbers of tests 
occur, and TT increases directly with the number of tests 
per component and inversely with component reliability. 
Intuitively one would feel that as TT decreases the accuracy 
of the computation procedure will decrease since fewer 
failures will occur, and less information will be obtained. 
One purpose of the simulation is to determine for what 
values of TT the various methods meet desired accuracy goals. 



2 

Borsting, Jack R., and Woods, W. Max, A Method for 
Computing Lower Confidence Limits on System Reliability Using 
Component Failure Data With Unequal Sample Sizes , Naval Post- 
graduate School, Monterey , California, June 196b. 
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IV. RESULTS AND CONCLUSIONS 



The analysis of accuracy results are shown in Tables I 
and II. The cases considered are grouped according to the 
number of components, system reliability, and value of TT. 
The component reliabilities, p^’s, are held constant in 
some cases while the number of tests per component, n^’s, 
are varied, to yield a range of TT values. In other cases 
the n^’s are fixed and the p^’s varied. Twenty-three cases 
of systems with reliabilities ranging from approximately 
.87 to .95 and consisting of 10, 20, and ^0 components are 
considered . 

As expected the tendency of all methods to increase in 
accuracy as TT increases is substantiated. In addition, the 
tendency of increased accuracy from methods A, to B, to C is 
also substantiated. Cases 21, 22, and 23 show clearly that 
for TT = 10.0 the accuracy of the three methods is essen- 
tially the same for all three methods of computation, since 
with this relatively high amount of testing the methods 
become identical. These three cases also show that as the 

number of system components changes, if R and TT are held 

s 

constant, the accuracy of the three methods remains essen- 
tially unchanged. 

It is up to the user to determine what accuracy is 
acceptable when employing any procedure to compute R g l( y )- 
For illustrative purposes, if one considers | R g - I < *02 

to be acceptable accuracy, then for a ^0 component system 
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with an overall reliability of approximately .90, the 

testing program should be such that TT = 6.50 is a minimum 

and TT > 9.0 would be desirable. As TT decreases below 

6.50 there is a marked deterioration in accuracy of all 

three methods as measured by both I R - A., I and the true 

level of confidence versus the desired level of confidence. 

Another key factor to consider in using any of these methods 

is that each is sensitive to the individual component test 

sample size, n^. If n^ is quite small, say less than 10, 

for any component regardless of TT, a failure on one of 

those n^ tests will tend to have a disproportionate effect 

on R s l( y )’ Note that in cases 6 through 10 accuracy proves 

to be not as good as in cases 1 through 5 for k = 20 and 

corresponding values of TT. This is because the n^ values 

are generally lower for cases 6 through 10 and in some 

cases n. is less than 10. 
x 

The results show little difference in accuracy between 
the 80 $ (y = .20) and 90$ (y = .10) confidence levels among 
the various cases and methods. There is no general tendency _ 
for the accuracy to be better at either level although one 
might expect that accuracy at the 80$ confidence level would 
be better than at the 90$ level. For example in case l^A 
(TT = 6.55) accuracy is better at the 90$ level, but in 
cases 13A (TT = ^.05) and 15A (TT = 9.10) accuracy is 
better at the 80$ level. 

These methods are approximations. No analytical justi- 
fication is given for the modifications suggested. Intuition 
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has been the guiding factor in development of the methods 
and the accuracy tests which are based on sound analytical 
techniques are the basis of acceptance or rejection of the 
method as a useful tool. Since systems vary widely in 
their composition, it is not possible to cover them all 
with a set of specific rules as to when it would be appro- 
priate to use one of the methods for computing R o 
However, from these results two general rules follow: 

1. For systems of ten to forty independent components 
TT greater than 10.0 should yield accuracies of 

j R g - A^_^ | < .01 provided that, 

2. no component is tested less than 10 times, i.e. 
n^ >_ 10 for all components. 

If these two rules are followed it can also be concluded 
that method B should be employed since there should be no 
appreciable difference in the accuracies of methods A, B, 
and C, and B is easier to compute than C. Also B does not 
suffer from the possibility, remote as it would be if TT 
were greater than 10.0, of yielding R g = 1.0 as would 

method A. Any system under consideration can be tested using 
using the computer simulation procedure, and as TT decreases 
below 10.0, either because of economic or time constraints 
in the testing program, this would be recommended. 
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of Accuracy Results by Case 
TABLE I . 
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NC indicates No Change from corresponding result of previous method. 

Summary of Accuracy Results by Case 
TABLE I. 
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*NC indicates No Change from corresponding result of previous method. 

Summary of Accuracy Results by Case 
TABLE I. 
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NC indicates No Change from corresponding result of previous method. 
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NC indicates No Change from corresponding result of previous method. 
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*NC indicates No Change from corresponding result of previous method. 
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"NC indicates No Change from corresponding result of previous method. 

Summary of Accuracy Results by Case 
TABLE I. 



•-q 

uqu<W 

ZD'Z:> 

C^OW 

HU-q 



VO 

CO 



in 



o 

53 



o 



m 

vo o 
oo s 



in 

oo 



Ln 



o 

53 



o 

53 



o 

53 



o 

53 



>o 

i 

r-H 

< 

I 

CO 

c 3 



>o 

l 



)o 



(0 



P 3 



^r 


CM 


vo 


in 


o 


o 


o 


o 



^r 


CM 


OA 


oc 


CM 


CM 


OA 


OA 



O 

C\J 



o 

on 

CVI 

OA 



o 

53 



o 

53 



CVI 

VO 

o 

o 



o 

in 

o 

o 



CM 

CM 

CM 



VO 

on 



o 

525 



o 

53 



CM 

oa 

CM 

oa 



o 

oo 

CM 

OA 



o 

o 

o 

o 



oa 

oo 



o 

o 

on 

o 



oo 



o 

cr\ 



o 

53 



o 

53 



-=r 

CM 



o 

53 



CM 

o 

O Oa 

S oo 



o 

CM 



o 

CM 



o 

CM 



o 

CM 



o 

CM 



o 

on 

CM 

cr\ 



o 

on 

CM 

CF\ 



oo 

t— 

oo 



oo 



oo 



oo 

t>- 

[>- 

oo 



H 

H 



o 

o 



OA 



o 

o 



OA 



o 

o 



CTA 



in 

CJA 



in 

oa 



in 

oa 

-E±- 



p 4 

xs 

£ 






w • 

CO o 
< 53 
CJ 



♦ o o o 
o oj^rcr 

| — | A l\ IN 



• AAA 

** i — I rH i — I 
i — I i — I CM || 

II II II -H 
•H *H *H 

A 

^OO 

OOOCh 

in o in oa 

i — I i — i • 

M ii II II 
•H *H «H «H 

C C C 0 



O 

-=T 



< 

in 



o 

CM 



VO 

A 

oo 

OA 

o\ 



•A • 

in o 
on_M- 



o "n 

#N 

. Ln 



rH vo 
cm on 

li ii 

' *H -H 



O- O 
CT\ OA 
cta oa 



o 

-=T 



PQ 

in 



o 

-=T 



CJ 

Ln 



^ OA 

in oa 

H°' - - 

II ii* II II 
•H — | -H *H 

c a a a 



o 

-M- 



< 

vo 



o 



PQ 

VO 



o 

-M" 



u 

vo 

I — I 



25 



*NC indicates No Change from corresponding result of previous method. 
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NC indicates No Change from corresponding result of previous method. 
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NC indicates No Change from corresponding result of previous method. 
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NC indicates No Change from corresponding result of previous method. 
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A 


B 


C 


16 


40 


1.95 


.8778 


.1222 


.0300 


NC 


6 


20 


1.45 


.9046 


.0954 


.0266 


.0238 


11 


40 


1.45 


.9230 


.0770 


.0497 


. 0220 


1 


20 


1.45 


.9512 


.0488 


.0147 


.0110 


17 


40 


2.60 


.8778 


.0080 


NC 


.0070 


7 


20 


2.95 


.9046 


.0495 


NC 


.0100 


12 


40 


2.95 


.9230 


.0617 


NC 


.0309 


2 


20 


2.95 


.9512 


.0258 


NC 


.0054 


18 


40 


3-90 


.8778 


.0458 


NC 


.0451 


8 


20 


4.05 


. 9046 


.0316 


NC 


.0249 


13 


40 


4.05 


.9230 


. 0420 


NC 


.0294 


3 


20 


4.05 


.9512 


.0179 


NC 


.0147 


19 


40 


6.50 


. 8778 


.0197 


NC 


.0193 


9 


20 


6.55 


.9046 


.0300 


NC 


.0266 


lit 


40 


6.55 


.9230 


.0166 


NC 


.0126 


4 


20 


6.55 


• 9512 


.0157 


NC 


.0115 
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Case 

No. 


k 


TT 




|Rs “ Ai_*| 


A 


B 


C 


20 


Ho 


9.10 


.8778 


.0033 


NC 


.0035 


10 


20 


9.10 


.9046 


.0269 


NC 


.0237 


15 


40 


9.00 


.9230 


.0064 


NC 


.0062 


5 


20 


9.10 


.9512 


.0138 


NC 


.0123 


21 


10 


10.0 


.9228 


.0051 


NC 


.0042 


22 


20 


10.0 


.9229 


.0052 


NC 


.0048 


23 


40 


10.0 


.9230 


.0051 


NC 


.0050 
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Case 
Ho *1 




TT 




|Rs - Ai_y| 


k 




A 


B 


C 


16 


40 


1.95 


.8778 


.0136 


NC 


.0124 


6 


20 


1.45 


.9046 


.0954 


.0532 


.0117 


11 


40 


1.45 


.9230 


.0447 


NC 


.0414 


1 


20 


1.45 


.9512 


.0488 


.0280 


.0088 


17 


40 


2.60 


.8778 


.0080 


NC 


.0070 


7 


20 


2.95 


.9046 


.0360 


NC 


.0243 


12 


40 


2.95 


.9230 


.0571 


NC 


.0420 


2 


20 


2.95 


.9512 


.0190 


NC 


.0138 


18 


40 


3.90 


.8778 


.0239 


NC 


.0234 


8 


20 


4.05 


.9046 


.0278 


NC 


.0213 


13 


40 


4.05 


.9230 


.0318 


NC 


.0295 


3 


20 


4.05 


.9512 


.0148 


NC 


.0129 


19 


40 


6.50 


. 8778 


.0055 


NC 


.0059 


9 


20 


6.55 


.9046 


.0246 


NC 


.0229 


14 


40 


6.55 


.9230 


.0198 


NC 


.0151 


4 


20 


6.55 


.9512 


. 0096 


NC 


.0088 



Summary of Accuracy Results for 'tf = .20 

Table II 



32 



Cas J 
11°. | 


k 


TT 




|Rs - 


A 


B 


C 


20 


40 


9.10 


.8778 


.0014 


NC 


.0016 


10 


20 


9.10 


.90^6 


.0215 


NC 


.0181 


15 


40 


9.00 


.9230 


.0052 


NC 


.0050 


5 


20 


9.10 


.9512 


.0115 


NC 


.0159 


21 


10 


10.0 


.9228 


.0062 


NC 


.0052 


22 


20 


10.0 


.9229 


.0061 


NC 


.0052 


23 


40 


10.0 


.9230 


.0062 


NC 


.0060 
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